[FEATURE] Ontology-guided extraction and typing - #539

Lexical Graph Coverage Report: The coverage is at unknown% (target: unknown%). Download the HTML report here.

Lexical Graph Coverage Report: The coverage is at unknown% (target: unknown%). Download the HTML report here.

Lexical Graph Coverage Report: The coverage is at 73.7% (target: 80%). Download the HTML report here.

AI-generated review. Reviewed manually before posting.

Smaller things, roughly in the order I'd care about them. Happy to open issues instead if you prefer.

Correctness

• entity_graph_builder.py:364,417 — new_query_var() is interpolated into the query text, and
  GraphBatchClient keys batches on the query string, so every typed value becomes its own single-row
  round trip. Measured with 50 attribute facts at batch_write_size=25: off → 2 round trips,
  subject → 52, both + include_local_entities → 104. A fixed variable name is safe here (the
  variable is local to a one-statement query).
• datatype_utils.py:227 — _coerce_datetime/_coerce_time strip the trailing offset and then parse, so
  it's discarded rather than applied, and coercion reports success. 2020-03-03T23:00:00-05:00 and
  2020-03-04T04:00:00Z are the same instant and store unequal; -05:00 and +09:00 are 14h apart and
  store identical. That defeats the range queries the feature exists for, and on Neptune a _date-suffixed
  key wraps the zone-stripped string in datetime().
• datatype_utils.py:231 — _coerce_datetime is the only coercer with no shape regex, so its accepted
  grammar is datetime.fromisoformat's, which widened in 3.11. '2020-03-03T09:30:00.5' coerces on 3.12
  and returns None on 3.10 (supported per requires-python), which at strict is the difference between
  keeping and dropping the fact. An _ISO_DATETIME regex would make the boundary yours.
• datatype_utils.py:242 — _coerce_time picks its strptime format from the colon count ('%H:%M:%S',
  no %f), so the (\.\d+)? group in _ISO_TIME is dead and every fractional-second xsd:time is
  rejected — while xsd:dateTime accepts and truncates the same fraction.
• datatype_utils.py:210 — _ISO_DATE admits a leading - and _coerce_date then does
  lstrip('-'), so -0500-01-01 (500 BCE) stores as 0500-01-01 and reports as conforming.
  _coerce_datetime refuses the same input, so the asymmetry looks unintended.
• ontology.py:537 — rdfs:subClassOf owl:Thing is a fatal OntologyLoadError, though _class_reference
  (:371) already exempts OWL_THING for domain/range. It's a legal, Protégé-emitted axiom, and the
  workaround (declaring owl:Thing a owl:Class) injects a meaningless Thing entity type into the
  prompt and the rendered class tree.
• ontology_index.py:137 — nothing checks that declared terms have distinct resolution_keys.
  schema:Person + foaf:Person loads clean; class_by_key['person'] is one of them and the other is
  unreachable by every name it has, while the renderers still name it. At strict those facts are dropped
  and reported as enforce_domain_range, pointing at a declaration that is correct. The
  label-vs-local-name shape is sharper: :Company rdfs:label "Organisation" alongside :Organisation
  makes the compliant emission resolve to the other class, and since create_entity_id hashes the
  classification, the two collapse onto one node id with a conflicting classIri — which also breaks the
  idempotence invariant documented at ontology_filter.py:20-25.
• naming.py:97 — render uses .upper(), fold uses .lower(), and they aren't inverses for names whose
  case mapping isn't fold-stable. rdfs:label "Größe" renders GRÖSSE, which resolves to nothing
  (index holds größe and groesse). Worse, Straße and Strasse both render STRASSE, so a compliant
  emission resolves to the wrong property and takes that property's datatype. casefold() plus indexing
  the rendered form would close it; the invariant test currently only covers local_name, ASCII.
• ontology_filter.py:446 — strict prunes statement.facts but leaves statement.value, and
  StatementNodeBuilder writes the __Statement__ node unconditionally, so the rejected assertion is
  still in the graph and in the statement vector index — the primary retrieval unit. Measured: statement
  survives with facts: 0. Worth either scoping the claim in the authority table or dropping/logging
  statements left with no facts. (topic.entities is never gated either.)
• neptune_graph_stores.py:174 — the widened except (TypeError, ValueError) still lets dateutil's
  OverflowError through: '99999999999999999999' under a _date-suffixed key raises out of
  create_property_assigment_fn_for_neptune, graph_construction.py:161-166 re-raises, and nothing in
  build_pipeline.py catches it, so the run dies mid-batch. Unlike the pre-existing metadata caller, this
  value comes from LLM output. Also the one production line this PR changes in the file, currently untested.
• neptune_graph_stores.py:170 — datetime() wrapping is decided by property-name suffix, ignoring the
  rdfs:range the call site already holds, and binds the raw literal rather than the normalised value.
• ontology.py:453 — datatype=ranges[0] takes the alphabetically-first of a sorted set with no
  multiplicity check, while _class_reference logs for exactly this ambiguity on the domain of the same
  object one line above.
• ontology.py:526 — range validation only checks startswith(XSD_NAMESPACE), so xsd:Integer /
  xsd:datetime / a bare namespace all load clean and are indistinguishable downstream from a real
  built-in whose text fallback is deliberate.
• ontology.py:355 — _values() stringifies literals and drops Literal.language, and _first_value()
  returns the alphabetically-first, so which language wins is decided by spelling, independently per
  predicate. A @de label beat an @en one in my test.
• ontology.py:275 — with vocabulary_format='turtle', a blank node referenced by more than one subject
  (a reused owl:Restriction) gets a fresh _:nXXXX label per process: three runs, three different
  blocks, three different sha256s. Since LLMCache keys on the formatted prompt
  (llm_cache.py:163-165), that's a 100% cache miss per chunk per run and no byte-reproducible build.
• prompts.py:303 — on the empty-constraints path with_ontology_constraints returns the template
  unchanged instead of clearing {ontology_constraints}, so a custom template using that documented
  extension point sends the literal brace group to the model.
• datatype_utils.py:259 — nine string-derived XSD types (NCName, ID, language, …) dispatch to
  _coerce_text, which returns any input unchanged, yet validates_datatype reports True for them
  because it's derived from _COERCERS.keys() — so the "unvalidated" warning is suppressed for types
  whose value space is never checked.
• ontology_filter.py:723 — when the model doesn't repeat the object string in its entity block, the
  parser yields a complement, resolution.object_class stays None, and _class_satisfies returns True
  for None, so the whole object side escapes both enforce_domain_range and enforce_entity_types at
  strict.
• ontology_filter.py:540 — no exclusion for the pipeline's own sentinel: resolution_key('__Local_Entity__')
  is 'local entity', so a declared :LocalEntity (or a "Local Entity" label) rewrites the sentinel at
  align and every downstream == LOCAL_ENTITY_CLASSIFICATION guard then misses.
• pipeline_utils.py:72,77 — the violation report only reaches spawn workers via _applied_logging_config,
  written exclusively by the new apply_logging_config, so a process that used stdlib basicConfig gets
  no report at all; and the dictConfig is passed as an initarg with no picklability check, unlike
  config_snapshot beside it, which pickle-tests each field and warns.

Security / robustness (author-supplied ontology, so low, but cheap)

• prompt_constraint.py:485 — rdfs:comment is interpolated verbatim, so a newline plus ## forges a
  section heading in the rendered block (reproduced: two ## Using this vocabulary headings), ``` escapes
  the turtle fence, and nothing rejects the | the response protocol uses as its field separator.
• datatype_utils.py:133 — _ANY_URI's ^\S+\.\S+\S*$ ends in \S+\S*; the trailing \S* is dead and
  only turns a linear rejection into a quadratic one (2/8/16 KB → 0.017/0.264/1.046 s).

Docs / hygiene

• ontology-guided-extraction.mdx:400, :81-83, :207-209 — the three "an off setting is a no-op"
  claims. ontology_authority='off' still rewrites {preferred_entity_classifications} from the ontology,
  and the placement table's 'off' row promises a byte-identical graph.
• graph-model.mdx:101-103 — the reserved-name guard is narrower than stated: typed_value is accepted at
  'subject' placement, and value is refused only at subject placements.
• The "Writing an ontology" section documents none of the load-time rules that will actually reject a file
  (every referenced class must be locally declared, owl:imports isn't followed, subClassOf cycles,
  dual-typed properties, missing/non-XSD ranges).
• ontology_filter.py:54 and two sibling comments assert an rdflib-free import boundary for extraction
  workers; a submodule import runs every ancestor __init__, and both ancestors import rdflib
  (confirmed: importing ontology_index alone leaves rdflib in sys.modules). The plain-data claim is
  true and worth keeping — it's just the import claim that isn't.
• lexical-graph/output.log — tracked, zero bytes, added by this commit; the only tracked *.log in the
  repo and not covered by .gitignore (which already ignores output.log under two other paths).

Method note: findings were reproduced against a real venv (rdflib 7.6.0, Python 3.12 plus a 3.10 for
the version-divergence item), not read off the diff.

Worked through all of these. Every one reproduced. Fourteen are fixed, nine are closed with a
reason, and one is worse than you measured.

Where you were too generous: _ANY_URI

You had 2/8/16 KB → 0.017/0.264/1.046 s. On my machine it grows 8× per doubling, i.e. cubic:

dotted prose + one space:  500 → 0.054s   1000 → 0.431s   2000 → 3.44s   4000 → 28.5s

And dropping the dead trailing \S*, which is what you proposed, removes the constant but
leaves ^\S+\.\S+$ quadratic — every candidate dot is still retried against every tail:

after dropping \S*:        2000 → 0.005s   8000 → 0.084s  16000 → 0.330s

So I replaced the alternation rather than patching it. The original language is exactly "contains
no whitespace, AND (has a scheme prefix OR has a . with a character on each side)", which is
three linear tests. Differentially verified against the original regex over 300,022 inputs — the
hand-written edge shapes plus 300k random strings over [A-Za-z0-9./:#?+- \t] — 0 mismatches.
64 KB now takes 0.0002 s. Worth flagging that the pathological input class is precisely the one
the check exists to reject, and the value comes from LLM output.

Fixed

Closed, with reasons

On the string family

Your framing was right and the fix was smaller than expected. Of the ten, only xsd:string is
genuinely unconstrained — returning the literal unchanged is the check for it. The other nine
have restricted lexical spaces that nothing here checks. Removing them from _COERCERS is the
whole change: they fall through to the unimplemented-type path, which already returns trimmed text
and already warns once per type, so stored values are identical and the user is simply told.

I chose not to implement the lexical checks. At strict a failing value is dropped, and losing a
fact because a token carried two consecutive spaces is a worse outcome than storing it
unchecked. These types document intent; they are not worth making into a gate.

Also fixed the validates_datatype docstring, which claimed True "for every XSD type with an
implementation, including the string family" — the clause that made this look intentional.

Docs and hygiene

• The three "off is a no-op" claims: corrected. 'off' still seeds
  {preferred_entity_classifications}, and the placement table's byte-identical claim now scopes
  to the emitted queries.
• graph-model.mdx reserved names: rewritten, including which placements the check runs at.
• Writing an ontology: new "What is checked when the file loads" subsection covering every
  load-time rule, owl:imports not being followed, and the two cases that load clean and do
  nothing.
• The rdflib-free import claim: corrected in three comments. You are right that the plain-data
  claim is the one that holds — confirmed that importing ontology_index alone leaves rdflib
  in sys.modules, because indexing.extract.__init__ re-exports the package.
• lexical-graph/output.log: untracked, and output.log added to .gitignore.

Two grammar decisions I left to Python

_coerce_datetime still has no shape regex, deliberately. I tested 3.10 rather than assume: the
divergence is confined to fractional-second digit counts other than 3 or 6, ISO basic format, and
week dates. Every form a model realistically emits behaves identically on both. Pinning that would
mean owning a date parser, so it is documented instead, including that a strict build on 3.10
can retain marginally fewer facts.

Thanks for the method note at the end of your comment — reproducing against a real venv rather
than reading the diff is why almost all of this was actionable without a round of argument.

Correction to my previous comment, on prompt_constraint.py:485.

I listed that item under "Fixed" without qualifying it, and only one of the three vectors you
named was actually closed. Flattening newlines stopped a comment forging a ## heading; the other
two still reproduced:

backtick-fence count in block: 3   (a single fenced block should have 2)
rendered line: Company  "A company|FOUNDED_YEAR|1994"

So a comment carrying ``` did close the fence early — putting the rest of the ontology and the
protocol section after it outside the code block — and | went through untouched.

Both now fixed in ea7b4e83:

• The turtle fence is sized to the content: one more backtick than the longest run in it, per
  CommonMark. A comment containing ``` gets a four-backtick fence, and the protocol section stays
  outside it.
• | in a description is replaced with /. It is the response protocol's field separator — the
  prompt asks for entity|RELATIONSHIP|entity and parse_extracted_topics splits emitted lines
  on it — so a | arriving from an rdfs:comment reads to the model as a worked example of the
  output it is being asked to produce, in the middle of the vocabulary it is being asked to use.
  A slash rather than a deletion, so a comment like "revenue|turnover" still reads as the
  alternation the author meant.

Neither is escaped, because there is nothing to escape to — this is prose fed to a model, not a
format with an escape syntax. Four tests cover it, including that an ordinary comment still gets a
plain three-backtick fence.

While I am correcting the record, one other qualifier I should have made explicit: on
ontology_index.py:137, "warns, naming the winner" is the whole of the change. The first-wins
resolution policy is unchanged, so the losing term is still unreachable by every name it has — the
warning tells the author to fix it rather than the code resolving it differently. If you think the
policy itself should change, that is worth its own discussion; I did not want it read as resolved.